In Defense of C4.5: Notes Learning One-Level Decision Trees

To appear in W. Cohen & H. Hirsh (eds.), Machine Learning: Proceedings of the Eleventh International Conference. (New Brunswick NJ, July 1994.) Morgan Kaufmann, San Francisco CA. We discuss the implications of Holte’s recentlypublished article, which demonstrated that on the most commonly used data very simple classification rules are almost as accurate as decision trees produced by Quinlan’s C4.5. We consider, in particular, what is the significance of Holte’s results for the future of top-down induction of decision trees. To an extent, Holte questioned the sense of further research on multilevel decision tree learning. We go in detail through all the parts of Holte’s study. We try to put the results into perspective. We argue that the (in absolute terms) small difference in accuracy between 1R and C4.5 that was witnessed by Holte is still significant. We claim that C4.5 possesses additional accuracy-related advantages over 1R. In addition we discuss the representativeness of the databases used by Holte. We compare empirically the optimal accuracies of multilevel and one-level decision trees and observe some significant differences. We point out several deficiencies of limited-complexity classifiers.